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(57) ABSTRACT 

A system for acquiring, processing, storing, analyzing and 
reporting data relating to the condition of a road or other 
pavement surface (102) in real time. The system includes a 
digital camera (104) mounted on a vehicle and positioned 
relative to a pavement surface (102) so as to capture images 
of the pavement surface (102) while the camera (104) is 
moving relative to the surface (102). The system also 
includes an illumination assembly (100) to illuminate the 
region from which an image is taken and an interface 
between the digital camera (104) and at least one computer. 
Hie system also includes a processor in the computer for 
processing the images to detect and classify cracks and other 
pavement surface (102) features. 

42 Claims, 10 Drawing Sheets 
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ROAD PAVEMENT DETERIORATION 
INSPECTION SYSTEM 

FIELD OF THE INVENTION 

The present invention relates to a road pavement crack 
detection system and apparatuses associated therewith and 
more particularly relates to a crack detection system utiliz- 
ing digital imaging for obtaining and storing data concerning 
the crack condition of a pavement. 

BACKGROUND OF THE INVENTION 

Cracking is the visible condition indicator most closely 
linked to pavement condition. Current methods do not detect 
cracking of the road network at the stage where early 
intervention would be cost effective nor attribute the inci- 
dence of cracking to a specific location (due to the small 
sampling). Major cost savings to authorities can be made 
from early intervention to seal finely cracked roads rather 
than waiting until the cracks develop to a more serious 
condition as is the case when using manual methods such as 
visual inspection. Finely cracked roads for instance can be 
sealed at a relatively low cost of $2-$4 per square meter on 
today's values compared to $30-$40 per square meter where 
cracking has progressed to a state requiring pavement reha- 
bilitation. 

There are a number of road crack measurement systems in 
use world wide. They range from passive systems such as 
video logging to active sensors such as lasers and ultrason- 
ics. Feasibility studies have monitored the availability and 
performance of these technologies and commercially avail- 
able systems. The existing systems are not capable of 
providing sufficient crack related data such as the ability to 
discern cracks on pavement surfaces including chip seal 
surfaces, at highway speeds. The existing systems further 
rely on post processing with manual intervention of image 
data in the office which often takes days before the data is 
available. Authorities have determined that there is a long 
felt want in this field of art to provide a system capable of 
delivering repeatable information on pavement cracking for 
common types of payment surfaces. As International Stan- 
dards recommend that cracking is a major indicator of 
pavement condition, there is a need for an objective method 
for assessment of the extent of cracking on the road network. 
As a result, research has turned to automated crack detection 
systems but to date, and prior to the invention disclosed 
herein, to the best of the applicant's knowledge, no other 
trader or authority has arrived at a suitable automated system 
which delivers the level of results required to overcome 
known problems associated with present crack detection 
methodology. Many roads with low traffic volumes are 
economically designed to have a thin bituminous surface. 
Thin cracked surfaces are particularly vulnerable in wet 
weather when traffic pumps water into the road pavement 
leading to pavement failures which causes a roughening of 
the surface and decreased asset value of the roads. Pavement 
durability is preserved by sealing the surface well before 
extensive cracking appears. 

Pavements such as roads, carriageways and the like are 
subject to heavy wear from traffic and degradation from 
sub -grade movement and deterioration in the pavement 
material itself. This necessitates regular monitoring of the 
pavement condition so the responsible government authority 
can plan maintenance programs and apply finances where it 
is most needed. This is a massive undertaking due to the 
usually extensive nature of road infrastructure. For efficient 
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monitoring every increment of road surface must be studied 
across its full width in order to detect serious and also fine 
hairline cracks which have the potential to develop into 
more serious cracks posing dangers for road users. Although 

s crack detection can be done visually by skilled personnel, it 
is almost impossible for visual inspections to obtain the 
necessary crack data for all sections of road in a particular 
municipality/region. The cost to do these inspections is 
prohibitive due to the amount of labour required and in any 

10 event such inspections suffer from the influence of human 
subjectivity and exposure to traffic hazards. 

Another more sophisticated method of crack detection 
involves use of a video camera which films the road surface. 
The video image is then post processed to detect the severity 

15 and extent of cracking. A crack resolution of 3-5 mm is 
about the best that can be achieved in the prior art methods 
by using algorithms to do extrapolations from the data which 
is read. 

Another method involves use of digital area scan cameras 

20 for achieving an image at high speed. The images from these 
cameras are prone to overlap and lack definition. 

Pavement condition monitoring generally involves mea- 
surement of a roughness profile and assessment of param- 
eters of pavement surface condition such as rutting, texture 

25 and cracking. Where cracks reach a certain size remedial 
surface treatments will not be an effective remedy so it is 
imperative that cracks are detected early and preferably 
when they are no larger than 1 mm. 

3Q The current manual methods of crack detection are time 
consuming and the extent of information gathered is limited. 
Manual methods are also capital intensive with poor return. 
There is clear need in the construction industry to arrive at 
more efficient ways of obtaining data concerning the con- 

35 dition of a pavement so that maintenance programs can be 
organized in the future and funds allocated. 

OBJECT OF THE INVENTION 

In order to overcome the shortcomings of the prior art 
methods the applicant determined that a survey vehicle 
40 could be equipped with automatic crack detection and 
classification systems using machine vision technology. This 
equipment would collect pavement condition data relating to 
cracking in conjunction with other equipment to measure 
road condition parameters such as roughness and rutting. 

45 SUMMARY OF THE INVENTION 

The present invention overcomes the shortcomings of the 
prior art by providing a crack detection system for accumu- 
lating and storage of data relative to the condition of a 

50 pavement. The invention further provides a system for 
obtaining digitized images of a pavement surface and com- 
puterized means for automatically analyzing the data 
thereby enabling efficient monitoring of the condition of the 
pavement surface. The present invention further provides a 

55 lighting assembly for use with a system for monitoring the 
condition of a road pavement surface wherein the lighting 
assembly enables illumination of fine pavement detail. The 
present invention further provides a digital camera for use 
with a system for monitoring the condition of a road 

60 pavement surface wherein the camera includes an automated 
exposure control enabling incremental snapshots of the 
pavement surface whilst the camera moves at up to and 
around 100 km per hour relative to the pavement. The 
invention further provides pavement monitoring software 

65 which stores, compiles and automatically assimilates data in 
"real time" relating to pavement condition obtained from a 
digital camera. 
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In broad terms, the system comprises a data acquisition In its broadest form the present invention comprises: 

vehicle having modular system equipment contained therein a system for enabling real time capture and processing of 

and which is capable of reading the condition of a pavement data re i at i ng to the condition of a pavement the system 

when the vehicle is travelling at speeds of up to 100 km per comnrisine* 

hour Whilst 100 km per hour is considered an appropriate s a ve hicle capable of traveling at highway speeds; 

speed it will be appreciated that the system has been . ,. , °, * . 

/ , , . j * r * i . q *l j * means in or on said vehicle for capturing pavement 

developed to accommodate taster speeds. Data trom the data , , ± + £ . r j \ e 

acquisition vehicle is obtained via the digital camera and is cra f A f and means for P^esstng and storage of 

processed through an image processing pipe line which ° a ? 1 Ure L a a * . _c 

resolves the images into a condition enabling data on the n Preferably, the pavement is a road surface and said means 

road surface to be read. This data is then converted into 10 for processing and storage of captured data includes means 

computerized reports and can be compared with previously f° r removal of unwanted data. 

read data to provide a comparative analysis of the condition ln another broad form the present invention comprises; 

of a particular length of pavement. a system for real time capture of and processing of 

In one broad form the present invention comprises a pavement crack data relating to the condition of a road 

system for obtaining, storing, comparing and analyzing data 15 pavement said system comprising; 

relating to the condition of a pavement surface; the system a vehicle capable of travelling at highway speeds; 

comprising; a digital image acquisition means for capturing images 

At least one digital image acquisition means positioned of said pavement whilst said vehicle is in motion; 

relative to a pavement surface so as to capture images processing means for receiving and anaylsing data 

of the surface whilst the digital image acquisition 20 captured by said camera and for removal of any 

means is moving relative to the surface, unwanted captured data, 

an illumination assembly to illuminate the region from There are a number of attendant benefits which attach to 

which an image is taken; tne S y St em according to the present invention and these 

an interface between the digital image acquisition means include reduced risk of injury to data collection personnel, 

and at least one computer 25 kss diversion of local res0 urces to manual data collection, 

processing means in said at least one computer for pro- reduced time losses, eliminating inconsistencies in manually 

cessing said images to detect and classify cracks and collected data, confidence in comparisons from year to year 

other pavement surface features. . t and region to region, confidence in accuracy of base data 

In another broad form the present invention comprises the used fof laQni and blldgeting rational basis for use of 

system as described above including: 30 craddn data ifl netWQrk * rformance> rti and main . 

a lighting assembly for illuminating a pavement surface lenance lanni reduced ^ at level> co u adng 

trom which digitized images are taken tor storage and data fof reporting and plannim , 

savings over current manual 

analysis according to the process herein described, the , „ „ r . . & , r . . * - 

J , * . . t r ^ data collection and savings in cost of maintenance and 

aS f « comprising, rehabilitation of road network, 
a light source; and 35 

means for providing even intensity and direction of the T)ninn nrcrniTmnxT ^ tr AVI7TXT/ -, 0 

i * a • r *u 1 1 c * j i_ BRIEF DESCRIPTION OF THE DRAWINGS 
light according to the level of exposure required by 

the digital image acquisition means. The present invention will now be described in more 

In another broad form the present invention comprises the detail according to preferred but non limiting embodiments 

system as described above wherein the digital image acqui- 4Q m & ^th reference to each form of invention and with 

sition means mcludes; reference to the accompanying drawings wherein: 

a digital camera, Pjq ^. g^^g a g enera t schematic layout of the arrange- 

said camera including a sensor, a time controlled auto- ment of ^ tem of ^ Qt iaventi 
matic external electronic exposure which captures 

images of the pavement surface whether the camera is AK FIG ' 2: shows an domination module reflector placed 
stationary or moving relative to the pavement surface 45 ^ res P ect to a camera and roadw *y ^ different views; 
and wherein the camera is responsive to reflectivity of FIG - 3: shows an example of image processing sequence 
the pavement surface. of a spray seal road surface acquired during trials. 
According to a preferred embodiment, the pavement FIG. 4: shows a schematic layout of system hardware; 
condition detection system is mounted on a vehicle allowing 5Q F IG. 5: shows a schematic layout of hardware including 
the digital image acquisition means to capture images of the da t a acquisition, control system and data processing hard- 
pavement surface. The camera includes an interface for ware* 

downloading images captured by the camera to a central ^ & sh(JWS a l t of the tem i ]e . 

processor which receives, stores, compares and analyzes mentation (real ume sub -sampling); 

data so as to provide via a user interface an up-to-date J 

picture of the condition of the pavement. 55 FIG ; 7: shows a thematic layout of system implemen- 

In its broadest form the present invention comprises: tatlon < real Ume continuous sampling long term); 

a system for real time capture and processing of pavement FIG * 8: shows a typical data acquisition vehicle showing 

crack data comprising; a possible equipment layout, 

a vehicle capable of traveling at highway speeds; 60 FIG. 9: Data logging sheet (raw data), 

means for capturing digital images of the pavement FIG. 10: Shows Road Condition Reports. 

surface; and 

means for processing and displaying captured data. A P*™!^ ° F P R ™RED 

Preferably, the digital image acquisition means comprises ^ ™™^^2R IMENTS ° F ™ E 

a digital camera and the processing means includes com- 65 einiiujn 

puter hardware and software for processing said digital The present invention provides a modular automated 

images. system for crack detection in a pavement and more particu- 
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larly detects finely cracked roads so that early repairs can be 
made to raise the overall condition of the road network. The 
automated crack detection system is contained in a vehicle 
which might be a truck or the like which is configured to 
enable all integers of the system to be contained within the 
vehicle. The parameters of the chosen vehicle are deter- 
mined according to the operating requirements of the sys- 
tem. At the very least, the vehicle must have electrical mains 
power mainly for illumination, an ability to scan a prede- 
termined width of pavement along with data capture systems 
to receive and process data captured by the automated crack 
detection system. Most of the electrical power is required for 
lighting so that a power generator located on the vehicle will 
be sized according to the power requirements per module. 
Other vehicle parameters include specified clearances 
between detection modules, internal space requirements for 
equipment, access for cabling and conduiting and a special 
configuration to suit user requirements. Each detection mod- 
ule of the preferred embodiment is designed to record crack 
data from a road imprint up to 750 mm wide and continuous 
in length in the direction of the vehicle motion. It is 
preferable that the conventional 25 meter truck width be 
maintained. 

Referring to FIG. 1 there is shown a schematic arrange- 
ment of an instrument configuration for the road crack truck. 
The instrument configuration is arranged into three areas, 
namely the vehicle, instrument cabin and driver's cabin. The 
key parts of this arrangement for implementation of the 
system are the vehicle and instrument cabin. Interaction 
between the vehicle hardware and software in the instrument 
cabin will be described in detail later. 

As can be seen from FIG. 1 the vehicle includes integral 
hardware being a camera set at a predetermined distance 
from a road pavement surface, a trigger means, means for 
generating power and lights for illumination of the surface 
to be imaged. 

In broad terms according to the system aspect of the 
invention there is provided a vehicle which is equipped with; 

an image acquisition module which includes, a digital 
camera and illumination assembly. 

The data captured by the camera is transferred into 
processing computers by means of a camera interface data 
distributor (CIDD). The data transferred to the computers is 
then processed by image processing software. 

IMAGE ACQUISITION MODULE 
(1) CAMERA 

The image acquisition camera includes a linear array 
sensor (which could include a photodiode, charge coupled 
device (CCD) or time delay integration (TDI) array of 
photosensitive elements) integrated into a camera. The sen- 
sor array is aligned transverse to the vehicle direction of 
travel. Camera, timing circuitry and an optical system are 
integrated with the sensor to record a millimeter of roadway 
per mm movement of the vehicle longitudinally. The number 
of light sensitive elements (pixels) in the linear array can be 
selected depending on the width of roadway to be inspected 
with a module (a number of image acquisition modules can 
be placed together to record large widths across a roadway). 
A 1024x1 pixel array was implemented in the prototype 
camera. The centre 750 pixels were used to create a 750 mm 
width image of the roadway per module. The rate at which 
the pixels can be read from the sensor has to be high (in the 
order of 40 Mhz) to allow images to be recorded at high 
vehicle speeds (up to 105 km/hr which equates to a line rate 
of up to approximately 29,000 lines per second. The sensi- 
tivity and quantum efficiency of the sensor was selected to 
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provide sufficient charge output above noise, but below 
saturation exposure, for a given exposure (light intensityx 
integration time) to the light reflecting from various types of 
road surfaces. Typical road reflectance was 4-30%. To 

5 minimize blooming effects within the sensor from exposure 
saturation, caused by highly reflective quartz particles in the 
pavement surface, for example, it was necessary to incor- 
porate anti-blooming circuitry. In addition, to minimize 
image blur at high vehicle speeds it was necessary to 

10 incorporate relatively short charge integration time capabil- 
ity (in the order of 1-20 microseconds) within the sensor. It 
was particularly important to maintain good image signal to 
noise characteristics at these sensor charge integration times. 
The correct exposure level was maintained by either varying 

is the integration time and or varying the position of an 8 bit 
quantisation window within the 10 bit analogue to digital 
sampling of the sensor charge output. This exposure contrdl 
methodology was based on digitally sensing the reflectivity 
v of the binder materials in aggregate road surfaces. The 

20 exposure was adjusted to yield consistent crack contrast 
from the surroundingjbinder material, independent of whit 
aggregate material (eg river gravel or blue-stone chip)»is^ 
^present*. Set camera lens apertures were used. A depth-of- 
field of the camera lens must be greater than the vertical 

25 movement due to the suspension of the vehicle is necessary 
if road features are to remain in focus. Typically, a lens 
aperture of transmission equal to 50% of full aperture was 
used. This provided a depth-of-field greater than 300 mm for 
a camera placed approximately 600 mm above the roadway. 

30 (2) ILLUMINATION 

In order to obtain digitized images of the pavement 
surface with high integrity, it was found important to pay 
particular attention to the illumination of that surface. Con- 
trast betwe en the c ra ck and road surface was maintained and 

35 fllumination configuration was setto remove any unplanned 
shadowing which could interfere with machine/computer 
interpretation of the image. Accordingly, the field of view of 
the camera is preferably illuminated evenly across the whole 
field. To achieve this, a cylindrical reflector was -used to 

40 focus linear quartz iodine-lamps on the road.surface. This 
ensured an appropriate-level of illumination for adequate 
exposure of the line scan array The design of the preferred 
embodiment provided evenjllumination across; the linear 
[arrayf In addition, the strip of focussed light provides a 

45 sufficient width of even illumination in the direction of the 
vehicle motion. This ensured the illumination strip did not 
move out of the camera's field-of-view as the vehicle moved 
up and down whilst travelling along the roadway, for 
example a c focussed fllummati^ 

50 30 mm was sufficient for a lmear.sensbr .fieldrofTview of the 
aoadway„oL750 mm. Cylindrical reflectors and lamps were 
placed before .and^^er_toev.c^era.TierativeJto-the-tmck 
■( directipnrThe angle that the illumination falls on the road 
surface is critical. It is possible that shadows can be mis- 

55 taken for crack data. To prevent this, the position of the 
lamps has been set to maximize crack contrast and minimize 
shadows cast from textured road surfaces. 

FIGS. 2(a) and (b) show alternative views of an illumi- 
nation assembly with respect to a camera and roadway. 

60 Referring to FIGS. 2(a) and (b) there is shown an illumi- 
nation module 100 which includes reflectors 101 and 105 
directed so as to illuminate a field of view on pavemenT" 
surface 102 relating to the line of sight of lO^of camera 104* 
Dotted lines 107(a) and 107(6) represent the focussed illu- 

65 mination -beam strip directed by reflectors 101 jmd 105* 
< respectively onto pavement surface 102. Informal operation 
it is^referable~fertKere _ be minimal movement between 
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illumination beams relative to the camera axis. Such move- 
ment might be caused by vehicle vibration and vertical 
vehicle movement. The object in providing specific focus- 
sing for the lamps with cylindrical reflectors is primarily to 
ensure adequate levels of illumination on the road surface 5 
and, secondly, to ensure that the level of illumination 
remains evenly distributed transversely and longitudinally 
with respect to the camera's optical axis, even when there is 
relative movement between the camera's field of view and 
the focussed illumination strips on the roadway. However, 10 
image processing software has been implemented to address 
and correct for uneven illumination. 

In the preferred embodiment of the present invention, the 
interaction between the lighting exposure control and image 
capture is important for the proper functioning of the system, 15 
The exposure control of the preferred embodiment is regu- 
lated by means of appropriate software architecture. Tests 
have shown that exposure times as short as 1.0 micro 
seconds are now possible with a specially adapted sensor 
whereas previously exposure times no less than 10 micro 20 
seconds were possible. At high speeds, where exposure 
times were no shorter than 10 micro seconds, image blur 
occurred effecting the quality of the road data captured. The 
operation of the sensor in conjunction with the image 
acquisition module and based on the available illumination 25 
has shown that images of 1 mm cracks can be obtained. In 
field trials the performance of the image acquisition module 
was evaluated by recording road images from six trial 
reference sites. These sites were selected for the variation in 
road surface types and the type, severity and extent of road 30 
cracking^ 15,000 trial images were recorded with a typical 
example depicting the image quality as shown in FIG. 3. 
FIG. 3 shows an example of an image on a spray seal road 
surface acquired during trials. 

According to the apparatus aspect the electronic exposure 35 
control of the digital camera is able to change exposure time 
for different road surfaces. For dark roads it is possible with 
appropriate exposure time to obtain good contrast between 
the road binder and cracks. 

(3) CAMERA INTERFACE DATA DISTRIBUTOR 40 

The line scan camera employed in the preferred embodi- 
ment allows high speed operation with good dynamic range 
and accurate exposure control but requires more elaborate 
interfacing than does a conventional area camera. The 
computing power necessary for real-time processing of the 45 
road images greatly exceeds the capacity of a single CPU, so 
several computers must be used to provide the level of 
performance required for the RoadCrack system. The image 
data must be appropriately formatted and directed to a 
processor when computing resources become available on 50 
completion of a previous operation. The Camera Interface 
Data Distributor (CIDD) board combines the functions of 
camera control and data distribution. 

The CIDD board performs several key functions in the 
system of the present invention, as follows: 55 

1. Measurement of vehicle speed using signals from a 
rotary quadrature encoder mounted on a wheel hub 

2. Setting of the control parameters for the camera soft- 
ware. 

3. Generation of a control signal to initiate a read of one 
line of image data from the 

camera for each millimeter of vehicle motion. This is 
termed the START signal. 

4. Generation of a control signal to allow the camera to 65 
integrate the electrical response to image intensity for 

a specified time. This implements an electronic expo- 
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sure system which allows very precise control of image 
data dynamic range. This signal is denoted by the label 
SHUT 

5. Provide appropriate signals to allow synchronization of 
multiple image acquisition/processing units to ensure 
identical sampling patterns in all sampled paths, 

6. Formatting of the camera data into frames of a size 
suitable for image processing. 

7. Marking each such frame with a unique identification 
code. 

8. Selection of frames for processing through use of a 
sampling table. 

9. Distribution of frame data to the various image pro- 
cessing computers. 

10. Monitoring of optical sensors to determine exposure 
parameters and verify correct system operation. 

11. Generation and transmission to the host computer of 
a stream of data packets, each recording all important 
information relating to a particular image data frame. 

12. Generation of a data stream to allow display of 
information on a LCD panel to assist the vehicle driver 
in maintaining the survey speed within the optimum 
range. 

The CIDD board consists of a main printed circuit board 
and a daughter-board with a CPU. The main board carries 
the connectors required to interconnect the camera and the 
three frame-grabbers used to feed data into the image 
processing computers. RS422 standard differential receiver 
chips accept two parallel 10-bit data streams from the 
camera. The low, middle or high 8-bits of the 10-bit data 
may be selected for processing. The desired 8-bit data is 
packed into a 16 -bit wide path, re -synchronized and trans- 
mitted in three identical streams using RS422 differential 
drivers. The daughter board carries a microprocessor which 
controls the CIDD operation, 1 Mbyte of static memory, an 
optional 1 Mbyte of ROM and a quad UART for serial 
communications support. 

Communications between the CIDD board and host 
image processor is performed over a RS232 serial line using 
a packet-based command protocol. The microprocessor on 
the daughter board runs a small multi- threaded operating 
system to allow it to perform its various functions efficiently 
and not limit the rate of data throughput of the complete 
system. The CIDD software is written in assembly language 
to maximize performance. An overview of the CIDD soft- 
ware is presented below. 

The operating system kernel supports multi-threaded 
operation using a simple scheduler. The scheduler allocates 
time-slices of 1/512 second and is invoked using an interrupt 
from the CPU periodic interval timer (PIT). A priority 
scheme is used to select the next task to be executed. A task 
may be suspended, if required, until a specific condition 
occurs. A thread may release the remainder of a time-slice if 
it cannot continue for any reason. Communications between 
threads is based on a simple event notification protocol. 
Shared resources are protected using semaphores whenever 
there is any potential for access conflict. 

The various threads are as follows: 

1. Each incoming command from the host computer is 
stored in a dynamically allocated packet buffer using a 
interrupt handler. The packets are stored in a queue for 
processing. The feed task maintains a reserve to two 
packet buffers for use by the handler since it is not 
possible to perform the memory allocation within an 
interrupt. 

2. The execpacket task performs the action 'specified by 
each command packet. A response packet, if required, 
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is placed in an output queue. The command packet is Channels 14 and 15 decode the quadrature signals from 

then modified to form an acknowledgment packet and the rotary encoder to give an accurate estimate of vehicle 

placed in the output queue. In addition, it transfers motion. 

frame data packets Channels 12 and 13 are used to synthesize quadrature 

3. The sendpacket task writes packets in the output buffer 5 signals, similar to those from a rotary encoder, and are 
out over a serial line to the host computer. intended for diagnostic purposes. 

4. Three tasks readport, writeport, and reportevent per- Channel 10 is used to support interrupt mode RTS/CTS 
form a similar function to the packet-based communi- ser i a l flow control over the internal MC68F333 serial inter- 
cations system, but are text-based to allow convenient f ace# 

software debugging over a serial port connection. 10 Channel 7 receives input from both an optical sensor and 

5. The camera 13 mon task manages the setting and veri- a user-operated switch. Either input may initiate acquisition 
fication of the camera software control parameters. It 0 f carne ra data at the beginning of a survey, or introduce a 
executes only at system initialization or when a change re f er ence mark in the packet communications data stream, 
to some parameter is required. Channels 8 and 9 are used to support synchronization of 

6. The display__mon task maintains a display of important 15 multiple CIDD units, 

information on a LCD display for use by the survey Optimum control of the camera electronic exposure is 

vehicle driver. facilitated through the use of a separate light sensor system. 

7. The exposure 13 mon task monitors the output of the The system comprises a sensor with a spectral response 
exposure sensor and computes the electronic exposure matching that of the camera. A lens is used to allow the 
interval best suited to the road surface characteristics. 20 sensor to sample light reflected from the region of road 

8. The status 13 mon task provides automatic control of the surface viewed by the camera. The raw sensor signal is 
illumination 'system to prevent road damage due to amplified, filtered and transmitted to the CIDD board where 
overheating when the vehicle stops. It also checks for a combination of analog and digital techniques are used to 
and reports abnormal conditions such as lamp failure. compute the optimum exposure interval for the observed 

The time -critical functions relating to camera timing, 25 road characteristics, 
frame formatting and data distribution are performed using The CIDD hardware and software can support reliable 

interrupt handlers and the Time Processor Unit (ITU) within image acquisition at vehicle speeds from 4 to 105 km/hr 

the microprocessor. The TPU consists of a block of config- when the image scale is such that one pixel corresponds to 

urable I/O logic, memory and an embedded CPU which are one millimeter on the road surface, 
organized to provide a wide variety of time-dependent I/O 30 
operations on up to 16 channels. The usage of the TPU for 
the CIDD board is described below. 

Channel 5 is used to produce the START pulse for the According to the system aspect of the invention there are 

camera, which initiates a readout of one image line. The ideally two ways of processing data obtained from the 

TPU Synchronized Pulse Width Modulation (SPWM) func- 35 CIDD. The first of these is by way of a pipe line principle 

tion is used. The pulse period is controlled to give one line and the second by means of parallel distribution of data 

per millimeter of vehicle motion. This channel is used as the through processing computers. Software algorithms used in 

primary synchronization source for much of the CIDD the processing of this data are the same irrespective of which 

operation. of the two data processing systems are adopted. 

Channel 6 uses the SPWM function to generate the SHUT 40 According to one embodiment the hardware/software task 

signal. The electronic exposure interval may be adjusted functionality operates according to the following steps: 
from zero to any required value in approximately 240 a ) data ^ acccptcd from the digital cam era; 
nanosecond (ns) steps^ b) framm &[ h ^ ated b me CIDD to form 

Channels 0, 1 and 2 are configured to produce a frame > { fof ; ssi 

trigger pulse just before the start of a new frame. The pulse 45 . . ., . ,7 f. , . c L • 
may be directed out of any one combination of the three c ) ; ma S e data 15 distabuted to each of the image process- 
channels. This pulse signals the frame grabber that the m S computers and to the host computer; 
following frame should be saved and processed. d ) the host computer issues commands to set up the 

A distribution table of up to 512 entries is used to control rec l uired Actions for image processing and data gen- 

the frame sampling strategy. Each table location of 8-bits 50 eration. 

specifies a required action. A pointer to the current table The interaction of the hardware between obtaining road 

entry is incremented after each frame and reset back to the crack data and lts stora S e assimilation and where applicable 

start after the last entry is reached. Bits 0, 1 and 2 indicate Nation is enabled by new software written specifically for 

the frame should be saved by frame grabber 0, 1 or 2. Bit 7, the Purpose, Operating system and software reside on a 

if set, forces a reset of the access pointer. Bit 6 may be used 55 central hard ^ attached t0 the host computer. The front 

to halt frame distribution after that entry is processed. This end s y stem ™ d processing units are diskless and boot 

distribution method allows complete flexibility in the choice £ om the !? ost machine over ^ n Ethernet without user infer? 

of sampling strategy vention. The front end system displays a user interface drl a / 

Channel 3 issues an interrupt at the start of the first line LCD-screen/touch screen^ which allows the user to ^ntroV 

in a frame. The interrupt handler then writes any required 60 J he sjfcteg fc^^n^Jggt and orocessmg eomputers. 
information for the last frame in a packet buffer and places / s y stem software implemented includes: 

that buffer in a queue for output by the execpacket task. — a)control of -system-start~up, initialization and control 

Channel 4 issues an interrupt at the start of the last line of parameters; 
a frame. The interrupt handler computes the average vehicle b) communication with the camera interface and data 
speed over that frame and updates the START period. 65 distributor - interpretation of commands; 

Channel 11 issues an interrupt which triggers generation c) communication with the user interface computer — 
of pulses on channels 0-2 at the appropriate time. interpretation of commands; 
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d) user interface to set up, control and diagnose system 
instrumentation; 

e) image and data acquisition and processing; 

f) frame grabber control; 

g) manage the distribution of image data processing load 5 
between parallel operating image processing comput- 
ers; 

h) apply the appropriate image processing algorithms to 
the image data; 

i) merge frame information with processing results and 
generate output data file. 

FIG. 4 shows a schematic layout of system hardware. 
FIG. 5 shows the hardware configuration categorized into 
data acquisition, control system and data processing. FIG. 6 
shows a schematic layout of the software system implemen- 
tation for real time sub-sampling. FIG. 7, shows a schematic 
layout of the software system implementation for real time 
continuous sampling. FIG. 9 shows a further general sche- 
matic layout of the arrangement of the system aspect of the 
present invention. 

IMAGE PROCESSING SOFTWARE I 

The first step of the image processing algorithm uses a 
"separable rank filter" to filter the image. A binary image is 
produced in which a pixel is ON if its value in the original 25 
image is lower than the corresponding value in the filtered 
image. 

In the second step candidates^ojj^lpjimt^ 
r v erse*cpac king«are^ method. 
To find the candidate for longitudinal cracking, a longitu- 30 
dinal shortest-path is found through the binary image using 
a dynamic programming algorithm. This is the top-to- 
bottom path with the most ON values. From this path a 
contiguous segment of a certain fixed length is selected, 
again maximizing the number of ON values. This is the 
candidate segment of longitudinal cracking, and the number 
of ON values is the "longitudinal cracking score" for the 
image. A candidate segment of transverse cracking, and a 
"transverse cracking score" for the image are then computed 
in the same way using a left-to-right shortest path, except 
that a "transverse texture score" is also computed. Whereas 
the "transverse cracking score" is based on the maximum in 
the right -most column of the distance-to-left image, the 
"transverse texture score" is equal to the minimum in the 
right-most column of the distance-to-left image. 

At each pixel on each candidate segment an integer crack 
width estimate is computed by inspecting the original image 
in the vicinity of the corresponding pixel. For each of the 
two candidate cracking segments, a smoothed median of 
these estimates is computed as the crack width estimate for 
the candidate segment. 

^TJ^pjg^ -is; determined; b 

comp^ng4^ 

The presence of transverse cracking is determined in a 
more complicated way: 

If the "transverse cracking score" is above a threshold, 
then transverse cracking is determined to be present. 
Otherwise, if the "transverse cracking score" is below 
another threshold, then transverse cracking is deter- 
mined not to be present. 
Otherwise, if the difference between the "transverse 
cracking score" and the "transverse texture score" 
exceeds a threshold, transverse cracking is determined 
to be present. 

Otherwise, transverse cracking is determined not to be 
present. 
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If longitudinal cracking is detected, but transverse crack- 
ing is not detected, the cracking type is "longitudinal". 

If transverse cracking is detected, but longitudinal crack- 
ing is not detected, the cracking type is "transverse". 

If neither longitudinal nor transverse cracking is detected, 
the cracking type is "none". 

If both longitudinal and transverse cracking are detected, 
or high density longitudinal cracking is detected, (see 
"Extent Measurement" below), the cracking type is "croco- 
dile". 

Extent Measurement 

An algorithm exists for calculating the average number 
and extent of cracks in both the longitudinal and transverse 
directions within the image. Probable crack locations are 
identified by finding local maxima in the distance array 
which is an intermediate image in the shortest-path calcu- 
lation. The results can be combined to estimate quantities 
such as density and extent of cracking in a frame. 

If the extent of longitudinal cracking exceeds a threshold, 
then "high density longitudinal cracking" is detected. 

Tyning Removal 

Approximately transverse tyning is attenuated in the 
image used for transverse crack detection by use of a rank 
filter with a window one pixel high. When no tyning is 
present, the standard algorithm (described above) can be 
written: 

Xl=transverse 13 rank(X0,nx) 

X2=longitudinal 13 rank(Xl,ny) 

B2=X0<X2 

transverse 13 detect(B2) 

longitudinal 3 detect(B2) 
where X0 is the input image, and nx and ny are, respectively, 
the transverse and longitudinal dimensions of the filter. 

When tyning removal is required the order is slightly 
altered so that transverse detection is carried out on an image 
which is produced by purely transverse filtering: 

X=transverse 13 rank(X0,nx) 

B1=X0<X1 

transverse 13 detect(Bl) 
X2=longitudinal 13 rank(Xl, ny) 
B2«X0<X2 

longitudinal 3 detect(B2) 

~ / 

Operator access to the system during routine data collec- 
tion activities is provided via a notebook computer, or 
"touch -screen" type device located in the front cabin of the 
host vehicle. The main functions of the user interface are: 
system initialisation and control 
selection from a list of data collection modes 
initiation of data collection, in a selected mode of opera- 
tion 

road location referencing and input of special features and 
events 

real-time processing of road cracking provided as opera- 
tor feedback 

Operation of the RoadCrack equipment produces records 
of road pavement cracking information referenced against 
an accurate distance reference determined from a measure- 
ment transducer. Selection of an optional road location 
referencing system produces a scrolling list of roadside 
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features which are synchronized with the distance measure- What is claimed is: 

meat record. Provision is made for a location referencing 1. A system for acquiring, processing, storing, analyzing, 

system specific to the needs of the road authority. Recording and reporting data relating to the condition of a road or other 

of data for intervals of 10 m, or shorter, being less than the pavement surface in real time; the system including; at least 

proposed 100 m reporting interval, allows data to be col- 5 one digital image acquirer positioned relative to a pavement 

lected in each direction of a roadway to be consistently surface so as to capture images of the pavement surface 

reported from a common referencing point. while the ima S e acquirer is moving relative to the 

The extent of operator intervention is minimized by pavement surface; and illumination assembly to illuminate 

provision of menu-prompts to minimize the subjectivity } h f «pon on the pavement surface from which an image is 

applied to the data collection process and thereby reduce the 10 taken ' an mtc rfacc bctween thc ac t qmrcr and a h f 

r *f . r o , c • » t one computer; a processor in said at least one computer for 

potential for error. System information is provided on the ^ dme F ^ of said - m t0 detect ^classify 

operating screen to allow direct operator response to extraor- Cfacks and Qther mem surface features 

dmary conditions encountered during the period of opera- 2 ^ tem ^ claimed in claim x wherein said interface 

tion. The user interface provides operator feedback which managcs thc manncr ia which ^ imagc acquirer 

includes an indication of the cracking determined from 15 acquires images. 

"real-time" processing. The provision of this information 3. The system as claimed in claim 2 wherein said interface 

allows the operator to record or respond to any anomalies or adjusts the acquisition timing of the digital image acquirer in 

problems which may corrupt the integrity of the data during relation to the speed at which said digital image acquirer is 

the period of operation of the RoadCrack system. A sum- moving relative to the pavement surface. 

mary report and data disk is provided at the end of eacrTdata 20 4. The system as claimed in claim 3 wherein said interface 

collection run - and~ no signfflcarit- further -processing is also adjusts the acquisition timing of the digital image 

required.to„produ_ce_more„detailed reporting formatsT acquirer relative to the reflectivity of the pavement surface. 

~ * 5, The system as claimed in claim 4 wherein said digital 

DATA SUMMARISING AND REPORTING f image acquirer is capable of receiving at least one line of 

rr, , 4 c t c it _ . ~ . . . i< image data for every 1 millimeter of movement of said 

The raw data format from the acquisition system consists Z5 . • i *• * -j , r 

r L r-j'-ji j digital image acquirer relative to said pavement surface, 

of a number of individual record types each denoted by £ iv * J i ■ j ■ c i - -i.e 

. . ^ n ~, . , x * • • r The system as claimed in any one of claims 2 to 5 

single letter prefixes. This data set contains information -j • . *ij i_ • c L 

^ • . r t* jt t r • wherein said interface includes a synchronizer for synchro- 
relating to Velocity, Exposure, Road Location Information, • • *u • •*• a • c • * 

if ^ !• • r . ' nizing the acquisition and processmg of images from more 

as well as Cracking summaries tor each 500 mmx750 mm 4U , . . 

* ii 7 a u *u * u enn -7m in man one digital image acquirer, 

image frame collected by the system. For each 500 mmx/50 JU n ^ , f . 1. , . - , . . r 

& . r .. . ... j - . i 7. The system as claimed in claim 1 wherein said interface 

mm image frame, the system reports the predominant crack- , „ *u a c j * * j * ui i 

/ . , ,. r . r ,. t x i( also manages the now ot data to said processor to enable real 

ing type (transverse longitudinal, crocodile or none), toe ^ si of said data 

cracking severity and crack width along with various other „ ^ B claimed ^ clajm ? wherein said ^ ^ 

values relating to the camera timing and processing param- . c , . , , , .. 

b 35 processing oi said data includes real time continuous 

processing, real time continuous processing by subsampling, 

This frame centric data is then summarized by combining ^ storage Q f sa i d data to aUow real ti me post-processing at 

frames occurring within each 10 meter longitudinal segment some later time 

into a single record. The method of summarizing the data 9 ^ system ^ c i aimed ^ c i aim 8 wherein said interface 

centres around recording the numbers of records that fall manages the flow of data to said processor by a number of 

into various categories. By recording the total number of steps including- 

image frames analyzed within the 10 meter segment, the / \ ^ i _ > A • . . u , e 

° 7 , , f ' (a) formatting each acquired image to a size suitable for 

number of frames recording each type of cracking can be nrocessinsr 

reported as a percentage of the total images analyzed. This x , . , . . -j .-^ , 

tu a ifi ui i + e (b) marking each image with a unique identification code; 

reporting method enables comparable results from succes- v 7 ^ & to ^ ' 

sive surveys to occur, even if the number of modules, or 45 , v .. . , . 

sampling methods change over time. £) debuting the images to the processor. 

, T7t _ . • i j . * . * n & t. 1«. The system as claimed in claim 9 including the 

when summarizing the data into 10 meter segments, the A .... i * * i u 1 c ■ j * 

- „ A . , , , . * . , additional step of selecting a subsample of acquired images 

frames are first categorized by predominant crack type and ^q^i^ 

by crack width. This becomes the intermediate Road Crack ,„ I i? I0 ^? Sl11 ^'* i • j - 1 • m L j 

t< . /^^x r j . nt 50 11. The system as claimed in claim 10 wherein said 

Data (RCD) format data file. , , . , , , u c , . . U1 

v / subsample is selected by means of a sampling table. 

Hie final reporting step involves further summarizing the u . The system as claimed in claim 9 wherein said unique 

RCD file into 100 meter segments, as shown in FIG. 10. For identification code contains information relating to the loca- 

each 100 meter segment, the total number of frames is tion at whicn me irnage was acqu i red . 

reported. This is followed by detailed breakdowns of the 55 13 ^ system as claimed m claim n wherein said 

number of frames exhibiting cracking in each of the three information on the location at which the image was acquired 

crack types along with the average crack widths for each. ^ at least partly provided t0 sa id interface by means of a 

Road Location Data and/or miscellaneous features entered Global p osi ti OD ing System (GPS) and/or a road location 

by the operator during acquisition are also reported in reference system. 

relation to the cracking data. 60 14 ^ system as claimed in daim t wherein said 

FIG. 8 shows a side elevation and plan view of a vehicle processor is capable of analyzing images of at least one type 

showing an equipment layout according to one embodiment 0 f road surface in real time. 

of the invention. 15. The system as claimed in claim 14 wherein said 

It will be recognized by persons skilled in the art that processor is capable of analyzing images of more than one 

numerous variations and modifications may be made to the 65 type of pavement surface in real time, 

invention as broadly described herein without departing 16. The system as claimed in claim 15 wherein said types 

from the overall spirit and scope of the invention. of pavement surface include: 
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i) sprayed seal; 

ii) asphalt; and 

iii) concrete. 

17. The system as claimed in claim 14 wherein said 
processor processes said images to detect and classify cracks 5 
and other pavement surface features by performing a num- 
ber of steps including: 

i) image filtering; 

ii) crack isolation; and/or 

iii) crack measurement. 10 

18. The system as claimed in claim 14 wherein said 
processor utilizes a number of algorithms to detect and 
classify cracks and other pavement surface features includ- 
ing: 

i) a rank filter algorithm; 15 

ii) a shortest path algorithm; 

iii) a cracking score algorithm; 

iv) a texture adjustment algorithm; and/or 

v) a cracking density algorithm. 2Q 

19. The system as claimed in claim 14 wherein said 
processor includes means for separating out artifacts and 
textures on the pavement surface. 

20. The system as claimed in claim 19 wherein said means 
for separating out artifacts and textures on the pavement 25 
surface includes means for the removal of or compensation 
for tyning and blooming from the images. 

21. The system as claimed in claim 14 wherein said 
processor classifies said cracks as: 

i) transverse cracks; 30 

ii) longitudinal cracks; 

iii) crocodile cracks; and/or 

iv) straight/non-cracked features. 

22. The system as claimed in claim 14 wherein said 
processor includes either a pipeline processor or a parallel 35 
processor. 

23. The system as claimed in claim 22 wherein said 
pipeline processor includes a series of circuit boards capable 
of analyzing said acquired image data continuously in real 
time. 40 

24. The system as claimed in claim 22 wherein said 
parallel processor includes a plurality of image processing 
CPUs arranged in parallel, each processing a subsample of 
the acquired data in real time. 

25. The system as claimed in claim 1 wherein said 45 
illumination assembly includes; a light source; and appara- 
tus for controlling the intensity and direction of light accord- 
ing to the level of exposure required by said digital image 
acquirer. 

26. The system as claimed in claim 25 wherein said 50 
apparatus for controlling the intensity and direction of light 
includes at least one reflector which concentrates the light 
emitted from the light source on an area of the pavement 
surface from which the digital image is taken. 

27. The system as claimed in claim 26 wherein said 55 
reflector is a cylindrical reflector and said light source is a 
linear quartz iodine lamp. 

28. The system as claimed in claim 1 wherein the digital 
image acquirer is a digital camera and includes a sensor; a 
time controlled automatic external electronic exposure 60 
which captures images of the pavement surface at a speed 
which is proportionate to the speed at which the digital 
image acquirer is moving relative to the pavement surface, 
and which takes the reflectivity of the pavement surface into 
account. 65 

29. The system as claimed in claim 28 wherein said sensor 
includes any one of: 
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a) a photodiode; 

b) a charged coupled device (CCD); or 

c) a time delay integration (TDI) of photosensitive ele- 
ments. 

30. The system as claimed in claim 1 wherein the system 
is mounted on a vehicle, allowing the digital image acquirer 
to capture images of the pavement surface as the vehicle 
moves along the pavement. 

31. The system as claimed in claim 30 wherein said 
vehicle is capable of travelling at highway speeds. 

32. The system as claimed in claim 31 wherein said 
highway speeds includes speeds between 5 kilometers per 
hour and 120 kilometers per hour. 

33. The system as claimed in claim 30 wherein said 
information on the location at which the image was acquired 
is at least pardy provided to said interface from information 
obtained from a distance transducer attached to a moving 
part of the vehicle. 

34. A system for real time capture and processing of 
pavement crack data comprising; a vehicle capable of trav- 
elling at highway speeds, a digital image capturer mounted 
in or on said vehicle for capturing digital images of the 
pavement surface; and a processor in communication with 
said digital image capturer, the processor for processing at 
least part of said captured data in real time, wherein the 
processor in communication with said digital image capturer 
includes a digital camera and the processor means includes 
computer hardware and software for processing said digital 
images. 

35. The system as claimed in claim 34 wherein said 
highway speeds includes speeds between 5 kilometers per 
hour and 120 kilometers per hour. 

36. Asystem for enabling real time capture and processing 
of data relating to the condition of a pavement, the assembly 
including; a vehicle capable of travelling at highway speeds; 
a pavement crack data capturer mounted in or on said 
vehicle for capturing pavement crack data; and a processor 
for processing captured data in real time. 

37. The system as claimed in claim 36 wherein the 
pavement is a sprayed seal, asphalt or concrete surface and 
said means for processing and storage of captured data 
includes means for removal of unwanted data. 

38. The system as claimed in either of claims 36 or 37 
wherein said highway speeds include speeds between 5 
kilometers per hour and 120 kilometers per hour. 

39. A system for real time capture of and processing of 
pavement crack data relating to the condition of a road 
pavement, said system comprising; a vehicle capable of 
traveling at highway speeds; a digital camera mounted in or 
on said vehicle for capturing images of said pavement while 
said vehicle is in motion; a processor mounted in or on said 
vehicle, and linked to said camera, for receiving data cap- 
tured by said camera, and for removal of any unwanted 
captured data in real time. 

40. The system as claimed in claim 39 wherein said 
highway speeds include speeds between 5 kilometers per 
hour and 120 kilometers per hour. 

41. The system as claimed in claim 1 wherein data is 
stored in any one of the following ways: 

i) no images are saved and only detection and classifica- 
tion results are saved; 

ii) only images are saved; 

iii) both images and detection and classification results are 
saved; or 
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iv) a sample of images and their corresponding detection 

and classification results are saved. 
42. A method for real time capture and processing of 
pavement crack data comprising the steps of: 

(a) using digital image acquirer having an electronic 5 
exposure control and positioned relative to a pavement 
surface to capture images of the surface while the 
digital image acquirer is moving relative to the surface, 

in conjunction with an illumination assembly which 
illuminates the region from which an image is taken; 10 

(b) using an interface between the digital image acquirer 
and at least one computer to manage the manner in 
which the digital image acquirer acquires images and to 
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manage the way in which the acquired data is passed on 
to an image processing CPU; 

(c) using software and/or hardware in an image process- 
ing CPU to process said images(s) of the pavement 
loaded into the computer by a number of steps which 
include: 

i) image filtering; 

ii) crack isolation; and/or 

iii) crack measurement; and 

(d) processing the images in real time and providing real 
time reporting of pavement crack data. 

***** 
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